Add FP8/FP4 block-quantized support to GatherBlockQuantized contrib op - #32480

Thanks for folding this into GatherBlockQuantized rather than shipping a second contrib op, and for the previous round: the WebGPU FP path is now reachable and locale-safe, the dead scale_size field is gone, the TArray rank-8 limit has a real message, the CPU FP path regained the per-thread row cache plus an incremental scale-index update instead of a full per-element decomposition, the unrelated MLAS/engram changes are split out, and the new BFloat16 / odd-nibble FP4 row / negative-index / defaults tests close the coverage gaps. All of my earlier threads are resolved.

Requesting changes on one blocker plus one semantic change that needs an explicit decision.

Blocking

The branch does not compile — every Linux, CUDA, TensorRT, WebGPU, ARM64, and minimal leg fails on the same error in contrib_ops/cpu/quantization/gather_block_quantized.cc (memset on MLFloat16 under -Werror=class-memaccess). Inline comments on both sites. Because this aborts the build early, more issues may surface once it is fixed.

Needs a decision

Out-of-range indices now zero-fill on CPU for the pre-existing integer types too. The CPU ORT_ENFORCE is replaced by a zero-fill, the schema text "It is an error if any of the index values are out of bounds" is deleted, and InvalidIndices flips from expect-failure to expect-success. That is not FP8/FP4-scoped — already-shipped int4/uint4/uint8 models go from a hard error to silently-zero output, diverging from ONNX Gather. It does make CPU match the pre-existing CUDA/WebGPU behavior, so there is a consistency argument, but it deserves an explicit owner sign-off and a mention in the title/description rather than riding along in a dtype-extension PR. Inline comment on the schema text.

Other suggestions (non-blocking)

1. ORT_ENFORCE is used for scales-shape validation at contrib_ops/cuda/quantization/gather_block_quantized.cc:217, two lines below an ORT_RETURN_IF_NOT in the same block. Prefer ORT_RETURN_IF_NOT for user-input validation so the failure comes back as a Status.

2. GatherBlockQuantizedParam grew by three TArray<int64_t> (192 bytes) that are passed by value into LaunchGatherBlockQuantizedKernel on every launch, including the integer paths that never read them. scale_broadcast_axis in particular only needs a flag per axis — TArray<int32_t> or a bitmask would halve/eliminate that cost.

3. docs/ContribOperators.md loses its trailing blank line at EOF in this diff. Please confirm it was regenerated by tools/python/gen_contrib_doc.py rather than hand-edited, otherwise the Doc Gen check will keep reverting it.

4. The WGSL LUT is emitted as const kFpDequantLut = array<f32, 256>(...) and indexed with a runtime u32. No other shader in the repo dynamically indexes a module-scope const array. Worth confirming Tint/Dawn accepts it (rather than requiring a var<private> array) once the WebGPU leg gets past the compile break.

5. Test gaps:

   • No FP8 case with a non-zero block_size where data_shape[quantize_axis] % block_size != 0. That is exactly the "reset the block index at every quantize-axis row boundary" case both the CUDA and CPU comments call out as the reason for the generic per-axis decomposition; FpSubRowBlockScale uses [1, 32] with block_size = 16, i.e. an exact multiple on a single row, so it never exercises it.
   • No scale broadcast on a middle axis (non-leading and non-quantize). FpGlobalPerTensorScale only broadcasts axis 0.

The current head addresses all substantive concerns from my previous review: the CPU half-output build failure is fixed, the out-of-range behavior is explicitly documented, CUDA validation and coverage are improved, and the WebGPU FP8/FP4 path is now registered and exercised with constant inputs. I also checked the final WebGPU lookup-table change: storing raw u32 bit patterns and bitcasting the selected runtime value avoids non-finite constant-expression failures without losing FP codes. The reported CI matrix is green. One non-blocking generated-doc whitespace cleanup remains inline.